Get Started

You can find our offical code here: TFB github

Table of Contents

  1. Quickstart
  2. Steps to develop your own method
  3. Steps to evaluate on your own time series
  4. Pipeline introduction
  5. FAQ
  6. Citing TFB
  7. Contact

Quickstart

Installation

From PyPI

Given a python environment (note: this project is fully tested under python 3.8), install the dependencies with the following command:

1
pip install -r requirements.txt

From Docker

We also provide a Dockerfile for you. For this setup to work you need to have a Docker service installed. You can get it at Docker website.

1
docker build . -t tfb:latest
1
docker run -it -v $(pwd)/:/app/ tfb:latest bash

Data preparation

Place the downloaded data under the folder ./dataset.

Train and evaluate model

We provide the experiment scripts for all benchmarks under the folder ./scripts/multivariate_forecast, and ./scripts/univariate_forecast. For example you can reproduce a experiment result as the following:

1
sh ./scripts/multivariate_forecast/ILI_script/DLinear.sh

Steps to develop your own method

We provide tutorial about how to develop your own method, you can click here.

Steps to evaluate on your own time series

We provide tutorial about how to evaluate on your own time series, you can click here.

Pipeline introduction

The below figure provides a visual overview of TFB’s pipeline.

  • The data layer is a repository of univariate and multivariate time series from diverse domains, structured according to their distinct characteristics, frequencies, and sequence lengths. The data is uniformly according to a standardized format.
  • The method layer supports embedding statistical learning, machine learning and deep learning methods. Additionally, TFB is designed to be compatible with any third-party TSF library, such as Darts, TSlib. Users can easily integrate forecasting methods implemented in third-party libraries into TFB by writing a simple Universal Interface, facilitating fair comparisons.
  • The evaluation layer offers support for a diverse range of evaluation strategies and metrics. And it also covers evaluation metrics found in other studies and enables the use of customized metrics for a more comprehensive assessment of method performance.
  • The reporting layer encompasses a logging system for tracking information, enabling the capture of experimental settings to enable traceability.
TFB Pipeline

FAQ

How to use Pycharm to run code？

When running under pycharm，please escape the double quotes, remove the spaces, and remove the single quotes at the beginning and end.

Such as: ‘{“d_ff”: 512, “d_model”: 256}’ —> {\“d_ff\“:512,\“d_model\“:256}

1
--config-path "rolling_forecast_config.json" --data-name-list "ILI.csv" --strategy-args {\"horizon\":24} --model-name "time_series_library.DLinear" --model-hyper-params {\"batch_size\":16,\"d_ff\":512,\"d_model\":256,\"lr\":0.01,\"horizon\":24,\"seq_len\":104} --adapter "transformer_adapter"  --gpus 0  --num-workers 1  --timeout 60000  --save-path "ILI/DLinear"

Citing TFB

If you find TFB helpful, please cite our paper (Bibtex below).

1
2
3
4
5
6
7
8
@article{qiu2024tfb,
  title   = {TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods},
  author  = {Qiu, Xiangfei and Hu, Jilin and Zhou, Lekui and Wu, Xingjian and Du, Junyang and Zhang, Buang and Guo, Chenjuan and Zhou, Aoying and Jensen, Christian S and Sheng, Zhenli and Bin Yang},
  journal = {Proc. {VLDB} Endow.},
  year    = {2024},
  pages   = {2363 - 2377},
  volume  = {17}
}

Contact

If you have any questions or suggestions, feel free to contact: